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SECTION  I 


INTRODUCTION 

The  work  reported  on  in  tL*s  document  is  the  result  of  a  mathematical 
approach  to  an  engineering  problem.  The  work  is  described  in  large  part  in 
mathematical  language.  The  purpose  of  this  section  is  to  describe  the  general 
Issues  involved  in  the  design  of  optical  correlation  filters.  This  will 
motivate  the  approach  to  the  engineering  problem  being  attacked. 

A  large  body  of  work  has  been  dedicated  to  shape-recognizing  or  object 
identification.  One  particular  approach  to  this  problem  is  optical 
correlation,  based  on  the  spatial  frequency  content  of  several  views  of  the 
object.  This  is  usually  attempted  in  a  holographic  optical  filter  system 
which  contains  sufficient  information  about  the  object  so  that  a  suitably 
transformed  image  of  the  object  can  be  processed  by  the  filter.  If  the  object 
sought  is  in  the  image,  a  large  correlation  peak  should  appear  in  the  filter 
output. 

The  class  of  correlation  filters  invented  by  Caulfield  and  Maloney, 
Reference  1,  and  extrapolated  by  Caulfield  and  Weinberg,  Reference  2,  Hester 
and  Casasent,  Reference  3,  and  Leger  and  Lee,  Reference  4,  termed  synthetic 
discriminant  functions  (SDF's),  hold  promise.  Very  roughly  speaking,  these 
filters  attempt  to  pack  a  great  deal  of  Information  into  one  device  by 
extracting  from  several  images  of  a  target  (the  training  set)  that  Information 
which  is  unique  about  each  of  them  and  combine  this  information  in  a  clever 
way  to  manufacture  one  filter  which  recognizes  each  image  in  the  training  set. 
However,  these  filters  reportedly  have  been  plagued  with  a  low  signal-to-noise 


ratio  (SNR  -  a  precise  Mathematical  definition  is  given  in  the  next  section): 
i.e.,  these  filters  have  no  problea  correlating  very  well  with  true  targets, 
but  very  often  give  high  (even  major)  correlations  with  false  targets.  In 
fa^c,  computer  simulations  run  by  the  author  on  realistic  data  show  that  the 
standard  recipe  for  constructing  SDF's  can  give  filters  with  an  SNR  of  less 
than  1.0,  even  against  a  training  set  of  images,  in  a  zero  background,  which 
are  edge-enhanced  and  energy-normalized.  These  low  SNR's  have  been 
independently  noted  by  Riggins  and  Butler,  Reference  5.  These  facts  make  the 
standard  recipe  for  constructing  SDF's  less  than  optimal. 

This  document  outlines  a  general  method  which  is  a  major  variant  of  the 
standard  recipe  for  manufacturing  SDF's  and  seems  to  solve  the  SNR  problem. 

In  fact,  when  applied  to  the  above  mentioned  data  set,  this  new  recipe  leads 
to  optical  correlation  filters  which  have  an  SNR  over  7.37  against  their 
training  sets.  Correlation  filters  with  such  SNR's  might  be  useful  for 
applications.  The  construction  of  these  new  filters  is  the  result  of  the 
formulation  of  an  engineering  problea  as  a  very  complex  mathematical  minlmax 
question.  It  is  somewhat  amusing  that  this  mathematical  ainimax  question  can 
be  interpreted  as  a  very  Involved  problea  in  plane  geometry.  This 
interpretation  also  makes  intuitively  clear  why  the  new  method  might  yield 
good  results.  The  mathematical  aininax  question  was  solved  for  a  fairly 
realistic  test  data  set  by  elaborate  computer  programs  which  took  a  great  deal 
of  di3k  space  and  hundreds  of  hours  of  CPU  time  to  run  on  a  VAX  11-780/VMS 
4.2.  The  long  running  time  needed  for  the  solution  of  the  minimax  question 
should  not  be  a  cause  for  alarm.  The  author  has  preliminary  ideas  which 


should  cut  the  CPU  time  for  the  solution  of  the  minlmax  question  down  to  no 


■ore  than  one-third  its  present  length,  even  using  the  present  computer 
configuatinn  available  to  him.  The  time  needed  for  the  solution  of  the 
■inimax  question  should  be  reduced  by  two  orders  of  magnitude  or  more  on  a 
better  configured  computer  system  (even  a  VAX)  with  more  physical  memory  and 
disk  space  available  for  the  computation,  using  smaller  (and  more  realistic) 
imagery  in  the  training  sets  and  even  more  advanced  numerical  techniques.  The 
computation  time  should  be  further  reduced  by  still  another  one  or  two  orders 
of  magnitude  on  a  really  advanced  machine. 

The  design  of  a  low  noise  (or  high  SNR)  correlation  filter  is  not  a 
straightforward  process.  Extreme  care  should  be  taken  to  insure,  to  whatever 
extent  possible,  that  the  filter  does  not  give  strong  positive  signals  to 
false  targets.  This  is  a  rather  complicated  task  since  one  does  not  know  a 
priori  all  possible  false  targets  -  and  even  if  one  did  know  them  and  there 
were  a  large  number  of  them,  just  how  would  one  take  all  of  the  information 
into  account  in  the  design  of  the  filter?  The  guiding  philosophy  behind  the 
work  in  this  document  is  that,  in  the  absence  of  any  a  priori  knowledge  of  the 
nature  of  false  targets,  one  should  design  the  filter  to  have  a  correlation 
output  plane  which  looks  as  much  as  possible  like  a  delta  function  centered 
over  the  target  when  the  target  is  centered  in  the  filter  input  plane. 
Furthermore,  in  the  design  process  care  should  be  taken  to  identify  all 
possible  independent  parameters  involved  and  optimize  the  design  by  varying 
these  parameters. 

If  one  adopts  the  design  philosophy  espoused  in  the  previous  paragraph, 
then  one  perhaps  will  design  correlation  filters  which  are  somewhat  sensitive 
to  perturbations  in  the  training  set  -  i.e.,  the  filter  perhaps  will  give 


■ajor  correlations  only  to  inages  which  are  quite  close  to  those  used  in  its 
training  set.  This  seems  to  be  a  necessarv  and  worthwhile  trade-off  if  one 
wants  to  have  filters  with  a  high  SNR.  This  will  be  an  especially  worthwhile 
trade-off  if  low  noise  SDF  filters  can  be  effectively  implemented  in  a  high 
speed  reusable  spatial  light  modulator.  Reference  6.  Then  one  will  have  a 
device  capable  of  displaying  several  hundred  SDF  correlation  filters  per 
second.  If  each  of  these  displayed  SDF  filters  can  be  manufactured  from  10  to 
100  training  images  and  still  have  a  high  SNR  one  will  have  a  device  which 
will  be  able  to  search  for  10°  to  1C;  images  per  second  with  a  low  probability 
of  committing  an  error.  This  perhaps  should  be  enough  for  practical 
applications,  especially  if  the  spatial  light  modulator  can  be  designed  to 
permit  minor  on-the-spot  programming  to  emphasize  those  SDF  filters  relevant 
to  current  weather  and/or  battle  conditions. 
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SECTION  II 


THE  MATHEMATICAL  APPROACH  TO  THE  PROBLEM 
SDF's  are  usually  manufactured  by  the  following  process.  One  starts  with 
two-dimensional  images  f  , . . . ,fn, . . . ,f  ,  often  called  the  training  set.  The 
analysis  of  this  section  is  entirely  general  and  does  not  depend  on  any 
particular  properties  of  the  f^'s  -  they  could  be  ink  spots,  for  example.  It 

might  be  intuitively  useful  to  view  these  f  's  as  either  infrared  images  which 
are  Inputs  to  some  filtering  device  or  as  square  matrices,  all  of  the  same 
size,  whose  entries  represent  intensity  levels.  It  is  useful  to  view  images 
as  (in  general  real  valued,  but  sometimes  complex  valued)  functions  on  the 
plane,  or.  if  they  are  large  matrices,  to  string  out  the  rows  of  the  matrix 
and  think  of  them  as  extremely  large  vectors  in  a  very  high  dimensional 

complex  inner  product  space.  Here  one  should  think  of  f . fR  as  lBages  of 

objects  one  is  seeking  and  f  j,...,f  as  Images  of  objects  one  is  not  seeking 

and  definitely  does  not  want  to  confuse  with  fj . fQ.  n  may  well  be  equal 

to  m.  An  SOP  h  is  also  an  image,  artificially  constructed  and  in  general 
complex,  in  the  author's  formulation.  Implemented  in  a  holographic  optical 
correlation  filter,  which  has  certain  properties.  At  the  very  least,  h  must 

satisfy  the  following  condition:  given  positive  numbers  A^ . A^  a  priori 

(which  are  not  necessarily  distinct,  and  in  fact  may  all  be  equal  to  the  same 
number,  1  say),  one  wants  the  Intensities  |<h,f^>|  to  satisfy 
| <h, f A> | 2  -  At2  (1  <  i  <  n) . 

Here,  and  in  general,  if  f  and  g  are  two  complex  images,  <f,g>  denotes  the 


(1) 


complex  Inner  product  between  the  two  Images,  viewed  as  complex  valued 

functions  on  the  plane  or  as  large  complex  vectors,  and  the  length  or  norm  of 

f  Is  given  by  | | f | j  -  <f , f>  .  ||f||  =  <f,f>  is  the  total  energy  in  the 

image  f.  The  condition  (1)  merely  demands  that  if  the  image  f^  is  centered  in 

the  input  plane,  then  the  center  pixel  of  the  output  correlation  plane  must 
,  2 

have  intensity  a^  . 

Before  doing  anything  else,  let  us  find  all  possible  solutions  to  the 
equations  (1).  If  further  constraints  or  requirements  are  later  placed  on  h, 
they  will  be  on  the  general  solution  of  the  equations  (1).  One  must  have 

<h,fi>  -  zJA1  (1  <  i  <  n)  (2) 

for  some  choice  of  complex  numbers  of  modulus  1.  This  gives  us  n 

simultaneous  linear  equations  which  h  must  satisfy.  Look  for  a  particular 
solution  hQ  of  these  equations  of  the  form 

b0 '  Vi . ,31 

for  some  choice  of  complex  numbers  a, . a  .  To  determine  the  a.'s,  plug 

in  l 

equation  (3)  into  the  equations  (2)  and  use  <f.,f  >  =  <f  ,f  >,  since  the  f  ’s 

1  J  J  1  K 

are  real  vectors,  to  get 

(<fi,fj>)(aj)  -  (ZjAj).  (4) 

If  the  n  by  n  matrix  (<f^,fj>)  is  nonsingular  (which  is  true  if  and  only  if 

the  vectors  f . ,f  are  linearly  Independent,  as  is  almost  surely  the  case 

l  n 

in  practice),  then  the  equations  (4)  uniquely  determine  the  aj's  and  thus  the 
hQ  which  satisfies  the  equations  (2)  and  equation  (3).  Let  h  be  any  other 
solution  to  the  equations  (2).  A  trivial  computation  shows  that  h'  -  h  -  hQ 
must  be  orthogonal  to  each  fj  (1  <  i  <  n). 

Thus,  one  may  obtain  every  solution  h  of  the  equations  (1)  as  follows. 


Choose  coaplex  numbers  z^ . zr  of  modulus  1,  determine  hQ,  as  in  equation 

(3) ,  which  solves  the  equations  (2),  by  solving  the  simultaneous  equations 

(4) ,  choose  h'  orthogonal  to  each  (1  <  i  <  n),  and  let  h  -  hQ  +  h' .  Ever/ 

SDF  h  must  be  of  this  general  form  independent  of  any  additional  constraint  or 
requirement  which  is  put  on  it.  This  general  setup  of  the  SDF  problem  is  the 
author's  own  variant  of  the  ideas  of  Caulfield  and  Maloney,  Reference  1, 
Caulfield  and  Weinberg,  Reference  2,  Hester  and  Casasent,  Reference  3,  and 
Leger  and  Lee.  Reference  4.  In  particular,  the  idea  of  choosing  h'  orthogonal 
to  the  f  *s  and  the  Zj's  to  be  general  complex  numbers  of  modulus  1  rather 
than  just  identically  1  (the  usual  recipe)  is  the  author's  own  idea  and  does 
not  seem  to  have  occurred  to  anyone  else.  The  importance  of  this  idea  will 
become  clearer  later  on. 

Some  notation  is  needed  to  describe  other  requirements  on  h  and  the  main 

problem  of  this  document.  For  any  x  in  the  plane  and  any  image  g,  let  gx(y)  = 

g(y  -  x)  for  any  y  in  the  plane.  If  one  initially  thinks  of  g  as  being 

centered  over  the  origin,  then  think  of  g  as  being  g  translated  so  as  to  be 

centered  over  x.  Ideally,  to  guarantee  a  high  SNR,  one  wants  that  the 

2 

measured  optical  intensities  | <h^ , f ^> |  all  be  maximal  for  x  the  origin  (note 

that  h^0  -  h)  and  that  | <hx • f ±> I  (1  <  i  i  ■)  be  as  small  as  possible  for 

all  x's  outside  of  some  a  priori  chosen  box  (or  any  other  region)  BA  about  the 

origin.  Here  is  empty  for  n+1  <  i  <  m.  The  term  SNR  is  now  defined 

precisely,  but  somewhat  arbitrarily,  as  follows.  The  SNR  of  h,  a  number 

intrisically  associated  with  h,  is  defined  to  be 

SNR(h)  -  min  SNR., 

1  <  i  <  a 

where  SNR^,  the  SNR  of  h  against  f^,  is  defined  by 


(5) 
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-  max  |<h  ,f  >|“/  max  |<h  . f.> 
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for  1  <  1  <  n  and 


x  <  B. 


SNR.  -  min  A  2 /max  |<h.f.>|2  (6b) 

1  <  J  <  n  x 

for  n  +  1  <  i  i  a.  Thus,  to  obtain  an  h  with  a  high  SNR  one  wants  to  Maximize 
SNR(h).  This  can  only  be  done  by  varying  the  z^s  and  h'.  A  procedure  to 
obtain  a  fairly  good  answer  to  this  Maximization  question  is  to  construct  h 
froa  those  parameters  and  h'  which  solve  the  somewhat  simpler  mlnimax 
problem 

o 

min  max  max  |<h  , f.>|  .  (7) 

Zj.h'  1  <  k  <  a  x  < 

This  is  the  author's  own  idea  and  own  analysis  of  the  problem.  The 
mathematical  and  computational  difficulty  involved  in  the  efficient  solution 
of  the  digitized  version  of  the  ainlaax  problem  (7)  cannot  be  overestimated. 
TLere  is  certainly  no  standard  piece  of  software  available  which  will  find 
even  local  solutions  of  the  ainimax  problem  (7),  let  alone  global  solutions, 
even  for  fixed  h' . 

Consider  the  ainimax  problem  (7)  in  which  one  fixes  h' ,  say  h'  >0.  Just 

why  would  one  expect  a  good  answer  to  the  ainimax  problem  (7)  merely  by 

varying  the  n  parameters  z^ . zr?  This  can  be  seen  by  reformulating  the 

question  as  one  in  planar  geometry.  Let  (d^)  be  the  n  by  n  matrix  which  is 

the  inverse  to  the  n  by  n  matrix  (<fi,fj>).  Then  for  1  <  i  <  n, 

a.  -  d.,z,A,  +  ...  +  d.  z  A  (8) 

i  il  1  1  in  n  n 


<h  ,f.>  »  a.<(f, )  ,f.>  +  ...  +  a  <(f  )  ,f.> 
x  k  l  lx  K  n  nx  k 

-  (d,,Z,A.  +  .  .  .  +  d,  Z  A  )<(f,)  ,f.>  +  .... 
'  11  1  1  In  n  n  lx  k 
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ziw  i_  ,  ♦ 

1  x. k , 1 
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where 


Vk.J  •  W'Vx’V  *•••*■  VfnW- 


(10) 


The  nuabers  w  are  coaplex  nuabers  (in  fact  real  nuabers)  and  aay  be 

x.k,  j 


viewed  as  vectors  in  the  plane.  The  nuabers  are  coaplex  nuabers  of  modulus 


1  and  so  aultlplication  by  thea  aay  be  viewed  as  a  rotation.  Hence,  these 
observations  show  that  the  solution  of  the  alniaax  problea  (7)  aay  be 
reforaulated  as  a  special  case  of  the  following  geometrical  problem:  given  q 


n-tuples  of  vectors  in  the  plane,  (vit . v,  ) .  (v  . . v  ).  find 

11  In  ql  qn 


rotations  Rj . Rn  so  that  the  aaxlaal  length  of  any  of  the  q  vectors 


R, v  +  ...  +  R  v.  .  R. v  +  ...  +  R  v  is  as  small  as  possible.  Note 

l  ii  n  in  l  q*  n  qn 


qn 


that  without  loss  of  generality  R^  can  be  normalized  to  be  the  identity. 


Alternatively,  one  may  view  the  alniaax  problem  (7)  as  a  problea  about  forces 


in  the  plane.  Think  of  each  Vjj  as  a  force  acting  on  a  point  particle  at  the 


origin.  One  then  wants  to  choose  the  rotations  Rj .  Rn  so  that  the  biggest 


net  force  on  the  particle  is  as  snail  as  possible.  For  a  hint  as  to  how 
difficult  this  problea  can  be,  merely  concoct  a  few  examples  when  n  =  3  and  q 
>3.  In  the  numerical  experiments  described  in  the  following  section  n  ■  36 
and  q  -  9,432,828. 

Engineers  and/or  physicists  perhaps  will  feel  soaewhat  uncomfortable  with 
reducing  the  design  of  a  high  SNR  SDF  down  to  the  minimax  problem  (7).  They 
perhaps  will  want  to  have  a  physical  interpretation  of  the  parameters  involved 
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in  its  solution  and  will  want  to  know  if  the  SDF  is  correlating  on  some 
particular  aspect  of  the  training  set  inapes.  These  questions  nay  have 
answers,  but  probably  not.  After  all,  it  is  an  easy  natter  to  give  sinple 
freshnan  calculus  optinization  questions  for  which  the  paraneters  involved  in 
the  solution  have  no  apparent  physical  Interpretation.  There  is  even  less 
reason  to  suppose  that  the  solution  to  the  enornously  conplicated  nininax 
problen  (7)  is  subject  to  a  physical  interpretation.  Furthernore,  asking  such 
questions  will  be  irrelevant  if  the  filters  given  by  the  purely  nathenatica 1 
solution  of  the  nininax  problen  (7)  perforn  as  desired. 
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SECTION  III 


RESULTS  OF  NUMERICAL  EXPERIMENTS 

Numerical  experiments  on  realistic  data  show  that  the  solution  of  the 
minimax  problem  (7)  can  yield  dramatic  improvement  in  the  SNR  properties  of 
SOF’s.  The  Images  used  in  the  following  experiments  were  36  256  by  256  real 
tank  images.  Even  though  the  recipe  given  in  the  previous  section  does  not 

depend  on  any  particular  property  of  the  training  set  f. .  f,  it  may  be  of 

i  n 

interest  to  describe  in  some  detail  the  exact  imagery  used  in  these 

experiments.  The  36  images  originally  were  512  by  512  images  in  a  cluttered 

background.  They  were  furnished  on  a  computer  magnetic  tape,  each  image 

2 

consisting  of  a  string  of  512  =  262,144  integers  between  0  and  255 

representing  intensity  levels  at  each  pixel  of  the  image.  The  tank  Images 
were  then  extracted  from  the  background  and  placed  into  256  by  256  arrays, 
using  DeAnza  array  processing  equipment.  The  original  images  had  been 
edge-enhanced  and  then  biased,  so  that  all  their  entries  were  nonnegative,  and 
then  each  pixel  was  discretized  into  one  of  256  equal  parts  -  hence  the  bit 
streams  which  appeared  on  the  data  tape.  A  somewhat  inexact  but  hopefully 
fairly  reasonable  way  to  recover  the  original  edge-enhanced  tank  images  alone 
was  then  employed  on  the  256  by  256  images  -  for  each  image  the  average  of  the 
nonzero  pixel  values  was  computed  and  then  subtracted  from  each  of  the  nonzero 
pixels,  leaving  those  pixels  with  0  values  unaltered.  These  edge-enhanced  256 
by  256  tank  images  were  then  energy-normalized  (i.e.,  they  were  divided  by 
their  length  to  make  them  into  unit  vectors.) 


Use  the  notation  of  the  previous  section.  In  the  following  calculations, 
n  -  a,  each  was  taken  to  be  an  11  by  11  box  centered  at  the  origin,  each  A ^ 
was  chosen  to  be  1,  and  h'  was  chosen  to  be  identically  0.  The  theory  of  the 
previous  section  is  not  affected  by  the  somewhat  ad  hoc  nature  of  the  choice 
of  h',  A  ,  and  B^,  and  the  programs  used  to  solve  the  minimax  problem  (7) 
arrived  at  their  answer  only  by  varying  the  z^'s. 

The  standard  way  of  making  an  SDF  is  the  process  described  in  the 
previous  section  in  which  one  takes  every  z^  to  be  1  and  h'  to  be  identically 
0.  Such  an  SDF  has  terrible  SNR  properties  against  this  training  set.  For 
example,  the  performance  of  this  h  against  f  gives  an  SNR  less  tha.'.  1.00, 
gives  1.02  against  fig,  and  against  fg,  f4>  fg,  fQ,  f?,  tQ,  and  f2g  gives  an 
SNR  of  less  than  1.25.  However,  the  author  has  used  the  recipe  of  the 
previous  section  to  construct  an  SDF  with  an  SNR  of  at  least  7.37  against  each 
f.  The  construction  of  this  latter  SDF  was  done  by  a  sequence  of  rather 
elaborate  computer  programs  which  took  a  great  deal  of  disk  space  and 
approximately  300  hours  of  CPU  time  to  run  on  a  VAX  11-780/VMS  4.2.  As 
explained  in  the  opening  section,  this  long  computation  time  is  not  really  a 
cause  for  concern. 

Note  that  16.00  is  certainly  a  very  plausible  upper  bound  for  the  SNR  of 
any  SDF  filter  h  made  from  energy-normalized  images  and  with  each  a^  equal  to 
1,  for  the  inner  product  of  h  with  each  training  image  quarter  is  a  complex 
number  whose  expected  modulus  is  approximately  1/4  and  so  whose  expected 
modulus  squared  is  approximately  1/16.  Practically,  an  SDF  with  an  SNR  of 
close  to  8.00  might  be  attained. 

The  author  recently  has  tested  the  performance  of  these  new  SDF's  versus 


the  standard  recipe  in  scenes  with  a  cluttered  background.  One  should  view 
these  experiaents  as  trying  to  find  a  real  tank  in  a  junkyard  of  tank  Darts. 

In  what  follows  the  tank  iaagery  used  is  the  same  as  the  tank  imagery 
referenced  above.  Select  one  of  the  36  256  by  256  tank  images  at  random  and 
place  it  in  the  middle  of  a  512  by  512  scene.  This  256  by  256  tank  image  is 
surrounded  by  twelve  128  by  128  blocks.  For  each  one  of  these  blocks,  select 
one  of  the  36  tank  images  at  random,  select  one  of  the  128  by  128  quadrants  of 
the  latter  image  at  random,  and  place  it  into  the  128  by  128  block  in  the  512 
by  512  image.  Do  this  for  all  12  128  by  128  blocks.  One  winds  up  with  an 
approximation  of  a  true  tank  located  in  the  middle  of  a  junkyard  of  randomly 
chosen  tank  quarters.  Examples  of  such  cluttered  scenes  are  given  as  Random 
Scene  1  in  Figure  1  and  as  Random  Scene  2  in  Figure  10.  In  these  scenes  only 
those  pixels  whose  magnitude  is  at  least  0.20  of  the  biggest  pixel  are 
illuminated.  The  author  computed  and  displayed  on  a  VHR19/6120  graphics 
terminal,  in  a  variety  of  manners  and  aspects,  the  correlation  plane  outputs 
from  an  SDF  filter  made  with  the  usual  recipe,  say  SDF1,  against  a  random 
sequence  of  junkyard  scenes  and  compared  them  to  the  correlation  plane  outputs 
of  an  SDF  filter  made  with  the  new  recipe,  say  SDF2.  Recall  that  these 
correlation  plane  calculations  done  via  an  FFT  are  actually  done  on  a  doughnut 
or  the  plane  tiled  with  these  512  by  512  junkyard  scenes,  so  there  was  a  good 
possibility  that  the  SDF's  could  get  confused.  This  happened  quite  often  with 
SDF1 ,  but  not  once  so  far  with  SDF2.  Approximately  200  independent  runs  of 
this  experiment  have  been  conducted  and  SDF1  gave  its  major  correlation  to 
clutter  in  the  background  approximately  25  percent  of  the  time.  SDF2  has 
never  given  a  false  major  signal  against  any  of  these  same  junkyard  scenes. 


In  fact,  in  no  instance  tested  so  far  has  SDF2  performed  worse  than  SDF1  and 


in  most  instances  has  performed  dramatically  better.  SDF2  so  far  has  always 

picked  the  true  tank  from  the  junkyard  and  has  given  a  very  sharp  recognition 

signal,  much  better  than  any  signal  coming  from  the  surrounding  clutter.  This 

13  12 

good  fortune  certainly  cannot  continue  for  all  of  the  approximately  36  x  4 
random  junkyard  scenes,  but  it  does  strongly  suggest  that  on  a  statistical 
basis  S0F2  performs  much  better  than  SDF1. 

Figures  2,  3,  6,  7,  11,  12,  13,  and  14  display  some  aspects  of  typical 
filter  output  correlation  planes.  Each  output  plane  is  a  512  by  512  array  of 
nonnegative  numbers  representing  the  correlation  intensities  given  by  either 
Son  or  SDF2  as  they  scan  Random  Scene  1  or  Random  Scene  2.  Think  of  each 
output  plane  as  representing  a  mountain  range.  Each  of  these  figures  is  a 
graph  which  gives  the  outline  one  would  see  by  viewing  the  mountain  range  in  a 
soith-to-north  direction  or  a  west-to-east  direction.  More  precisely,  each 
figure  is  512  pixels  wide,  and  the  height  of  the  graph  above  each  pixel  is  the 
maximum  intensity  one  would  see  along  the  column  corresponding  to  the  pixel  if 
the  view  is  south-to-north  or  is  the  maximum  intensity  one  would  see  along  the 
row  corresponding  to  tlu  pixel  if  the  view  is  west-to-east.  In  all  of  these 
figures  the  signal  coming  from  the  tank  in  the  cluttered  scene  is  the  central 
spike  and  has  height  1.  The  largest  height  in  each  picture  is  displayed  as 
750  pixels  high.  Figures  2  and  3  show  that  SDF1  gives  a  false  major  signal  to 
clutter  in  Random  Scene  1,  and  Figures  6  and  7  show  that  SDF2  does  not  make 
such  a  mistake.  Figures  11  and  12  show  that  SDF1  gives  a  very  large  false 
signal  to  the  clutter  in  Random  Scene  2  whereas  Figures  13  and  14  show  that 
SDP2  has  no  trouble  distinguishing  the  true  tank  from  clutter.  Figures  2,  3, 
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11.  and  12  are  typical  of  what  goes  wrong  with  SDF1  in  25  percent  of  these 
tests.  Figures  6,  7,  13,  and  14  are  characteristic  of  SDF2  in  every  instance 
so  far  tested. 

Horner  and  Gianino,  Reference  7,  have  recently  suggested  using  as 
correlation  filters  the  phase-only  versions  of  SDF  filters.  These  filters  are 
■anufactured  by  starting  with  an  SDF  filter  h,  taking  its  Fourier  transform 
F(h),  saving  only  the  phase  of  F(h),  say  P(F(h)),  and  then  taking  as  one's 
filter  F  1(P(F(h))).  They  report  considerable  improvement  in  the  output 
correlation  plane  with  these  phase-only  filters  versus  that  produced  by  the 
SDF  filter  alone.  The  author  manufactured  the  phase-only  versions  of  SDF1  and 
SDF2,  say  P0SDF1  and  POSDF2,  and  tested  their  performance  against  a  sequence 
of  randomly  generated  junkyard  scenes.  Unfortunately,  both  POSDF1  and  P0SDF2 
performed  very  unsatisfactorily  in  most  tests,  giving,  for  Instance,  a  sharp 
major  correlation  to  empty  portions  of  the  scenes  in  several  instances.  In  no 
instance  tested  has  P0SDF1  or  P0SDF2  performed  as  well  in  target  recognition 
as  SDF2.  Figures  4,  5,  8,  and  9  represent  the  output  correlation  planes  of 
POSDF1  and  POSDF2  against  Random  Scene  1  and  are  typical  of  their  performance 
in  Just  about  every  instance  tested.  In  these  figures  the  maximum  intensity 
is  displayed  as  750  pixels  high,  and  the  intensity  given  by  the  filters 
against  the  true  tank  is  not  a  set  level,  as  opposed  to  the  situation  with 
SDF1  and  SDF2 ,  where  the  intensity  given  by  the  filters  against  the  true  tank 
is  1.  There  may  well  be  overwhelming  advantages  to  phase-only  filters  over 
the  recipe  described  in  this  document.  However,  the  author's  conclusion  from 
these  tests  is  that  the  recipe  described  in  this  paragraph  certainly  is  not 
the  way  to  make  them. 
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The  author  also  tested  the  performance  of  S0P1  and  S0F2  against  a  sharp 
false  signal  by  examining  the  correlation  plane  outputs  of  the  two  filters 
against  an  input  scene  consisting  of  a  spike  of  height  1  at  the  origin.  In 
buch  instances  the  signal  from  the  origin  was  practically  buried  in  noise  in 
the  background. 

These  experiments  give  considerable  hope  that  correlation  filters 
manufactured  with  the  recipe  detailed  in  this  document  will  give  superior 


performance  in  practice. 


v*> 
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Figure  14.  SDF2  vs  Random  Scene  2,  W-E  View 
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SECTION  IV 


OTHER  ISSUES,  WORK,  AND  APPROACHES 

Some  sort  of  edge-enhancement  is  necessary  on  a  training  se*  of  images  if 

one  wants  a  high  SNR  correlation  filter.  The  usual  simple  example  given  to 

justify  this  assertion  is  that  a  uniformly  filled  in  square  and  a  uniformly 

filled  in  circle  of  roughly  the  same  size  will  correlate  quite  well  with  one 

another,  even  though  they  are  quite  different  objects.  This  would  not  be  true 

if  they  were  reduced  only  to  their  edges.  This  example  shows  that  some  sort 

of  edge-enhancement  is  necessary  for  the  training  image  in  matched  filters, 

let  alone  for  the  training  images  in  the  much  more  complicated  SDF  filters. 

Given  that  some  edge-enhancement  is  necessary,  then  shouldn't  more 

edge-enhancement  be  even  better?  If  the  tentative  answer  is  yes,  then  one 

plausible  way  to  obtain  filters  whose  output  correlation  plane  resembles  a 

delta  function  centered  over  the  target  when  the  target  is  centered  in  the 

filter  input  plane  is  to  make  the  filter  from  a  training  set  of  images  which 

are  extremely  edge-enhanced.  This  train  of  thought  leads  to  difficulties. 

Edge-enhancement  operators  should  work  roughly  as  follows.  View  an  image  as 

the  graph  of  a  function  f  on  the  plane.  Let  D  and  D  be  the  standard  partial 

x  y 

derivative  operators  along  the  x  and  y  axes.  It  is  plausible  that  D^lf)  and 

Dy(f)  should  be  small  on  the  interior  of  uniform  objects,  but  should  be  large 

when  crossing  the  boundary  of  a  uniform  object.  Hence,  if  P(D  ,D  )  is  any 

x  y 

constant  coefficient  differential  operator  without  constant  term  on  functions 

of  two  variables,  then  there  is  a  good  chance  that  P(D  ,D  )(f)  somewhat 

x  y 


resembles  f  with  the  interior  of  uniform  objects  eliminated  but  with  edges 
enhanced.  There  is  a  strong  plausibility  argument  that  every  true 
edge-enhancement  operator  must  be  of  this  form.  Let  E  be  any  edge-enhancement 
operator.  Then  E  should  be  a  rule  wLlch  assigns  to  an  image  f  another  image 
E  ( f )  with  the  properties:  (i)  E  is  linear  (E(af  +  bg)  =  aE(f)  +  bE(g)  for  all 
images  f  and  g  and  constants  a  and  b);  (ii)  E  is  continous  (if  f  is  close  to 
g,  then  E(f)  should  be  close  to  E ( g ) ) ;  (iii)  E  annihilates  the  constants  (E(C) 
=  0  for  any  constant  image  C);  (iv)  E  is  translation  invariant:  and  most 
importantly,  (v)  E  is  local  (E(f)(x)  only  depends  on  the  values  of  f 
arbitrarily  close  to  x).  If  one  admits  these  plausible  assumptions  for  an 
edge-enhancement  operator  E,  then  it  is  an  easy  exercise  in  the  Schwartz 
theory  of  distributions  (Hoermander,  Reference  8)  that  E  must  be  a  constant 
coefficient  differential  operator  without  constant  term.  Such  an 
edge-enhancement  operation  might  be  implemented  optically,  for  it  is 
equivalent  to  multiplication  by  P(-ik1>-ik2)  in  Fourier  transform  space. 

Notice  that  after  any  operation  of  this  sort,  the  Fourier  transform  of  the 
processed  image  vanishes  at  the  origin.  For  example,  if  P  is  the  Laplacian 
Dx  +  Dy  ,  then  P  is  equivalent  to  multiplication  by  -|k|  in  Fourier 
transform  space.  Hence,  there  are  limits  to  how  much  edge-enhancement  one  can 
do  without  greatly  distorting  the  original  image,  for  all  edge-enhancement 
operators  Involve  eliminating  or  suppressing  low  spatial  frequencies,  thus 
emphasizing  the  high  spatial  frequencies  and  putting  a  great  deal  of  weight  on 
the  noise  in  the  scene.  This  suggests  that  even  a  matched  filter  would  not 
perform  very  well  if  it  were  made  from  a  training  image  which  was  extremely 
edge-enhanced.  Perhaps  this  difficulty  can  be  removed  by  truncating 


P(-ik, ,-lk0)  at  a  certain  level  outside  of  some  disk  about  the  origin.  An 
extreme  case  of  this  Is  multiplication  by  1  -  8^  in  Fourier  transform 
space,  which  will  be  referred  to  as  minimal  edge-enhancement.  The  problem 
with  this  truncation  operation  is  that  multiplication  by  such  a  truncated 
polynomial  in  Fourier  transform  space  would  no  longer  be  a  local  operation 
when  transferred  to  physical  space  -  i.e.,  it's  effect  on  a  given  pixel  would 
depend  on  other  pixels  quite  far  away  and  not  just  on  arbitrarily  close  ones. 


This  seems  to  contradict  one’s  intuition  as  to  what  properties  an 
edge-enhancement  operator  usually  should  have. 

It  is  quite  possible  that  the  recipe  for  SDF's  given  in  this  document 


will  make  moot  all  questions  as  to  what  kind  and  how  much  edge-enhancement 
should  be  done  on  a  training  set  of  images.  Numerical  experiments  conducted 
on  filters  made  with  the  recipe  outlined  in  this  document  give  filters  with  a 
quite  high  SNR  on  training  sets  with  only  minimal  edge-enhancement  -  only  a 
small  disk  about  the  origin  in  Fourier  transform  space  need  be  set  to  0, 
perhaps  even  just  the  origin  itself,  in  order  to  obtain  filters  with  a  high 


SNR.  Numerical  experiments  on  this  important  question  are  continuing.  If 
they  are  as  successful  as  early  experiments  indicate,  then  the  recipe  given  in 
this  document  will  automatically  eliminate  the  questions  as  to  what  kind  of 


edge-enhancement  and  how  much  edge-enhancement  is  enough  on  a  training  set  in 
order  to  get  a  low  noise  correlation  filter.  The  answer  will  be  only  an 
absolutely  minimal  amount.  After  all,  the  major  reason  for  doing  any 
edge-enhancement  at  all  is  to  construct  high  SNR  correlation  filters. 

There  is  another  important  technical  advantage  to  using  Images  in  the 


training  set  which  are  at  least  minimally  edge-enhanced.  If  each  image  in 
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the  training  set  is  edge-enhanced,  then  at  the  very  least  this  implies  that 
the  Fourier  transform  F(f^)  vanishes  at  the  origin,  or  equivalently  that  the 
average  pixel  value  av(f^)  is  0.  Since  h  is  a  linear  combination  of  the  fj's, 
the  average  pixel  value  av(h)  is  alSw,  0.  Now  let  g  be  any  constant  image  - 
i.e.,  every  pixel  value  of  g  is  some  fixed  constant  C.  But  then  <h,g>  - 
Cav(h)  =0,  so  h  will  give  zero  correlation  to  any  constant  image.  Note  that 
if  av(h)  were  not  0,  then  Cav(h)  could  be  quite  large  and  h  would  give  major 
correlations  with  obviously  false  targets.  It  is  for  this  reason  that  at 
least  a  minimal  amount  of  edge-enhancement  be  done  to  the  images  in  the 
training  set  for  h.  On  the  other  hand,  if  av(h)  =  0,  then  for  any  image  g 
which  is  the  same  size  as  h,  <h,g>  ■  <h,(g  -  av(g))>.  Note  that  the  image 
g  -  av(g)  is  a  minimally  edge-enhanced  version  of  the  original  image  g  since 
av(g  -  av(g))  -  0.  This  is  important  in  practice,  for  suppose  the  SDF  h  is 
made  from  a  training  set  of  images  which  are  32  by  32  in  size  and  minimally 
edge-enhanced.  If  h  is  searching  for  targets  in  a  512  by  512  scene  g' ,  then 
even  if  g'  is  extremely  edge-enhanced,  its  resulting  generic  32  by  32  subscene 
certainly  need  not  have  even  its  average  pixel  value  0.  But  the  previous 
analysis  shows  that  if  h  is  made  from  a  training  set  of  weakly  edge-enhanced 
images,  then  no  edge-enhancement  at  all  need  be  done  on  g'  for  the  filter  h  to 
behave  as  if  each  32  by  32  subscene  was  weakly  edge-enhanced. 

Many  practitioners  of  this  subject  insist  that  the  images  used  in  the 
training  set  of  an  SDF  be  energy-normalized.  Energy  normalization  is 
equivalent  to  replacing  each  image  f^  by  f ^ / 1 1 f A 1 1 .  80  that  the  total  energy 
in  each  training  image  is  1.  (One  should  first  edge-enhance  and  then 
energy-normal ize  the  image.  The  two  operations  should  be  done  in  this 
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particular  order,  for  they  do  not  comaute.)  The  usual  reason  given  for  doing 

this  is  to  reduce  the  variation  in  the  cliaatic  effects  in  which  the  targets 

are  located.  This  is  plausible  since  Stefan's  Law  iaplies  that  the  total 

4 

aaount  of  radiation  eaitted  by  a  black  body  is  proportional  to  T  .  most  of 
which  is  in  fairly  low  (infrared)  frequencies  for  normal  temperatures.  Thus 
small  changes  in  the  temperature  of  a  black  body  result  in  rather  high  changes 
in  the  amount  of  infrared  radiation  it  emits.  Furthermore,  since  one  a  priori 
has  no  reason  to  know  which  aspect  of  a  target  one  will  encounter,  it  is 
plausible  that  each  image  used  in  the  training  set  should  be  normalized  to  be 
of  the  same  energy  to  initially  give  each  training  image  equal  weighting  in 
the  design  of  the  filter. 

The  initial  conditions  (1)  were  introduced  so  that  the  SDF  filter  would 
work  if  employed  in  a  threshold  detector.  In  order  for  such  a  threshold 
device  to  work  well  It  probably  must  be  necessary  to  optically 
energy-normal ize  the  input  scenes  to  the  filter.  There  are  at  least  two 
reasons  why  this  practice  should  be  done  with  a  certain  amount  of  caution. 
Since  the  SDF  will  be  made  from  energy-normalized  images,  say  32  by  32  pixels 
in  size,  it  will  be  looking  for  energy-normalized  images,  32  by  32  pixels  in 
size.  But  an  energy-normalized  512  by  512  image  certainly  does  not  in  general 
have  each  32  by  32  subscene  proportionally  energy-normalized.  If  the  object 
sought  is  the  brightest  object  in  the  512  by  512  scene,  then 
energy-normalizing  the  entire  image  will  leave  the  target  subscene  more  than 
proportionally  energy-normalized,  and  no  harm  will  result  if  one  is  merely 
searching  for  the  largest  correlation  peak.  But  if  there  is  a  much  brighter 
object,  say  a  fire,  in  the  upper  left  hand  corner  of  the  image,  and  the  target 


is  in  the  lower  right  hand  corner,  then  energy-normalizing  the  entire  scene 


perhaps  will  make  the  image  of  the  object  sought  so  faint  as  to  be  useless. 

The  probability  of  this  unfortunate  effect  will  surely  be  reduced  if  one  uses 
correlation  filters  with  the  highest  possible  SNR,  for  the  possible  higher 
correlation  given  by  an  intense  false  target  will  be  partially  or  totally 
offset  by  the  higher  SNR  of  the  filter.  Even  extreme  cases  of  this  phenomena 
can  be  eliminated  if  it  is  feasible  to  optically  energy-normal ize  those 

subregions  of  the  imput  plane  which  are  the  size  of  the  training  images  used 

to  construct  h. 

It  must  be  noted  that  there  is  a  difficulty,  though  a  low  probability 
one,  with  this  SDF  correlation  method.  The  Schwarz  inequality  implies  that 
the  SDF  will  have  largest  correlation  with  unit  vectors  that  do  not  look  like 
the  targets  in  the  training  set,  but  instead  look  like  the  SDF  itself.  The 

difference  might  be  quite  pronounced  since,  to  a  crude  first  approximation. 
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the  length  of  h  is  approximately  m  if  there  are  m  edge-enhanced  and 
energy-normalized  images  in  the  training  set  for  h.  For  the  SDF  filter 

process  to  really  work  with  a  low  probability  of  error,  one  must  make  an  act 

of  faith  that  there  are  very  few  real  objects  which  look  more  like  the  SDF 
than  the  target  images  themselves. 

There  are  many  papers  in  the  literature  on  the  manufacture  of  SDF's  whose 
philosophy  is  as  follows.  Given  a  training  set  of  images,  try  and  find 
(usually  by  some  sort  of  orthogonalizatlon  process)  those  Images  in  the 
collection  which  are  the  important  ones  and  make  an  SDF  from  these.  The 
proponents  of  this  method  find  this  philosophy  to  be  plausible  by  claiming 
that  the  eigenvalue  list  of  the  matrix  ( <f ^ , f j> )  contains  one  or  two  of 


comparable  size  and  that  all  others  are  incredibly  small  in  comparison.  It 
must  be  emphasized  that  this  is  true  only  if  the  f  's  are  not  edge-enhanced. 
Here  is  a  very  crude  engineering  argument  for  this  fact.  Suppose  that  the 
f^'s  are  energy-normalized.  Since  they  are  not  edge-enhanced,  then  the  images 
fj  are  solid  blobs  of  unit  energy.  Since  this  is  the  case,  <fi(fj>  *  1  to  a 
first  approximation,  and  so  (<f^,fj>)  *  an  n  by  n  matrix  of  l's,  to  a  first 
approximation.  It  is  a  simple  matter  to  check  that  the  eigenvalue  list  of  the 

latter  matrix  is  n,  0,  0 .  0.  Recall  that  it  has  been  emphasized  earlier 

in  this  section  that  one  must  use  a  training  set  of  images  which  are  at  least 
weakly  edge-enhanced. 

Now  suppose  that  the  images  f^  are  edge-enhanced  and  energy-rormalized 
and  let  us  again  employ  a  crude  engineering  argument  to  determine  a  first 
approximation  to  the  eigenvalue  list  of  ( <f ^ , f j> ) -  Since  the  ( s  are 
edge-enhanced  and  energy-normalized,  they  resemble  blzarrely  twisted 
coathangers  suspended  in  space  with  unit  energy.  Hence,  <fj,fj>  *  1  and 
<f^,fj>  a*  0.  to  a  first  approximation,  if  i  *  j.  Hence,  to  a  first 
approximation,  in  this  case  (<fj,fj>)  *  the  identity  matrix,  and  the 
eigenvalue  list  for  the  latter  matrix  is  1,  1....,  1.  In  any  event  this 
argument  suggests  that  all  of  the  eigenvalues  for  (<fitfj>)  have  approximately 
the  same  order  of  magnitude,  and  that  most  if  not  all  of  the  images  f ^  are  of 
equal  importance.  This  is  confirmed  in  actual  practice  by  at  least  one 
example  -  for  the  36  edge-enhanced  and  energy-normalized  tank  images  used  in 
the  numerical  experiments  of  the  previous  section,  the  eigenvalues  of  the  36 
by  36  matrix  (<f^,fj>)  range  between  0.6  and  2.2.  Thus,  attempts  to  make  a 
good  SDP  by  choosing  the  6  most  important  out  the  36  edge-enhanced  and 


energy-normalized  tank  images  are  definitely  doomed  to  failure.  After  all, 
there  is  something  suspicious  about  a  philosophy  which  asserts  that  one  can  do 
a  better  Job  at  recognizing  images  by  throwing  away  information  about  them. 
These  qualitative  statements  were  v„*’ified  in  an  objective  manner  through  very 
extensive  numerical  experiments  carried  out  by  the  author  during  the  summer  of 


SECTION  V 


CONCLUSIONS  AND  RECOMMENDATIONS 

Extensive  numerical  experiments  strongly  suggest  that  applying  the  recipe 

given  in  this  document  to  a  training  set  of  weakly  edge-enhanced  and  then 

energy-normalized  images  yields  an  SDF  filter  which  gives  very  sharp 

recognition  signals  to  the  training  set  images  and  has  a  very  low  probability 

of  committing  an  error.  Such  filters  might  well  prove  to  be  useful  if  they 

can  be  implemented  in  a  high  speed  reusable  spatial  light  modulator,  thereby 

3  4 

storing  information  about  10  to  10  training  set  images,  particularly  if 
input  imagery  to  the  filter  can  always  be  globally  and  sometimes  locally 
energy-normalized. 

The  numerical  experiments  detailed  in  this  report  were  only  carried  out 
on  one  set  of  tank  imagery.  These  experiments  should  be  repeated  for  many 
other  sets  of  imagery  to  see  if  high  SNR  SDF  filters  can  be  made  from  them  by 
the  recipe  given  here.  Furthermore,  the  successful  implementation  of  these 
filters  in  a  real  optical  system  can  only  be  done  via  computer-generated 
holograms.  Whether  or  not  this  can  indeed  be  done  should  be  intensively 


studied. 
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